Facial recognition frictionless access control

ABSTRACT

An access control system can include a camera that detects an image regarding a subject, a database maintaining information, a facial recognizer that receives the image and identifies a facial feature of the subject based on the received image and the facial information maintained by the database, and a processor to alert security or another user when an authorized user accessing an area of a building is under duress, when an unauthorized user is accessing an area of a building by tailgating after an authorized user, when an authorized user has not provided two-factor authentication, or when an authorized user is in a particular area of the building.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a Divisional application of Ser. No. 17/057,476,filed Nov. 20, 2020, which claims priority to U.S.C. § 371 NationalPhase Application of PCT Application No. PCT/US2019/033111 filed May 20,2019, which claims priority to U.S. Provisional Application Ser. No.62/674,476, entitled “FACIAL RECOGNITION FRICTIONLESS ACCESS CONTROL”and filed on May 21, 2018, which are expressly incorporated by referenceherein in its entirety.

BACKGROUND

Access control systems can be used to selectively enable users to passthrough specific locations in a building or other facility. Accesscontrol systems may include card readers, cameras, or other systems thatreceive input information and determine whether to provide access byopening doors, gates, or other impediments. While typical access controlsystems provide a level of automation and security to buildings,additional security or verification is needed in some situations.Accordingly, improvements in access control systems are desired.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

According to an implementation of the present disclosure, an accesscontrol system can include a camera configured to detect an imageregarding a subject. The access control system can also include adatabase configured to maintain facial information and skeletalinformation regarding a plurality of subjects. The access control systemcan include a processor in communication with the camera and thedatabase, wherein the processor is configured to: receive the image andidentify a facial feature of the subject based on the image and thefacial information maintained by the database; receive the image andidentify a gesture of the subject based on the image and the skeletalinformation maintained by the database; and determine whether thesubject is under duress based on at least one of the facial feature andthe gesture. The access control system can further include an alarmdevice configured to output an alarm based on an indication of duressfrom the processor.

According to another implementation of the present disclosure, an accesscontrol system can include a camera configured to detect an imageregarding a subject. The access control system can also include adatabase configured to store images and information corresponding to aplurality of subjects. The access control system can further include aprocessor in communication with the camera and the database, wherein theprocessor is configured to: detect a face in the image; and encode theimage by degrading an encoding of the image at a region corresponding tothe face.

According to another implementation of the present disclosure, an accesscontrol system can include a camera configured to detect an imageregarding a subject. The access control system can also include adatabase configured to store images and information corresponding to aplurality of subjects, a corresponding location of each subject of theplurality of subjects, one or more messages associated with at least oneof the plurality of subjects, and a corresponding message stateassociated with each subject, wherein each message state indicateswhether or not one of the one or more messages is assigned to thecorresponding subject. The access control system can further include aprocessor in communication with the camera and the database, wherein theprocessor is configured to: identify the subject based on the detectedimage; determine that the message state associated with the subjectindicates a message of the one or more messages is assigned to thesubject; and cause at least one of a mobile device associated with thesubject or a notification device in a vicinity of the subject to outputthe message.

According to another implementation of the present disclosure, an accesscontrol system can include a camera configured to detect an imageregarding a subject. The access control system can also include aprocessor in communication with the camera and configured to: identifythe subject based on the image and identifies at least one additionalsubject based on the image; determine movement information regarding thesubject and the at least one additional subject; and determine whetherthe at least one additional subject has permission to move through anaccess point based on the identified subject, the identified at leastone additional subject, and the movement information.

According to another implementation of the present disclosure, an accesscontrol system can include a camera configured to detect an imageregarding a subject. The access control system can also include a datacommunications device in communication with a mobile device associatedwith the subject. The access control system can further include adatabase configured to maintain facial information regarding a pluralityof subjects. The access control system can also include a processor incommunication with the camera, the data communications device, and thedatabase, wherein the processor is configured to: execute anidentification algorithm to identify the subject based on the detectedimage and the facial information maintained by the database; determinethat the identification algorithm was not successful in identifying thesubject; and perform one or more second factor authentication processesbased on the identification algorithm being not successful.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed, and this description is intended to include all suchaspects and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements, andin which:

FIG. 1 is a block diagram of an access control system according to anaspect of the present disclosure;

FIG. 2 is a flowchart of an example method for detecting duress usingfacial recognition and skeletal tracking according to an aspect of thepresent disclosure;

FIG. 3 is a flowchart of an example method for face redaction usingfacial recognition and region of interest encoding according to anaspect of the present disclosure;

FIG. 4 is a flowchart of an example method for intelligent subjectdetection according to an aspect of the present disclosure;

FIG. 5 is a flowchart of an example method for managing intelligenttailgating according to an aspect of the present disclosure; and

FIG. 6 is a flowchart of an example method for second factorauthentication in an access control system according to an aspect of thepresent disclosure.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well known components may be shown in blockdiagram form in order to avoid obscuring such concepts.

The present disclosure relates generally to the field of access controlsystems. More particularly, the present disclosure relates to facialrecognition frictionless access control. As discussed herein, typicalaccess control systems lack some abilities which can allow a personentrance into an unauthorized area of a building. For instance,additional security or verification is needed in some situations, suchas where typical systems are unable to detect a user under duress,provide a second security verification, or provide notification of thepresence of another person in certain areas of the building.Accordingly, the present disclosure describes access control systems forproviding an increased level of security, compared to a typical accesscontrol system. In an aspect, the access control systems of the presentdisclosure can alert security or another user when an authorized useraccessing an area of a building is under duress, when an unauthorizeduser is accessing an area of a building by tailgating after anauthorized user, when an authorized user has not provided two-factorauthentication, or when an authorized user is in a particular area ofthe building. Further, the present disclosure includes a frictionlesssystem, which means a system that makes it as seamless as possible for asubject seeking access to provide the authentication needed to gainaccess. In other words, a frictionless system may include a systemhaving no perceived delay or physical action required by the subject.For example, a frictionless system may allow a door, entryway, or gateto open before the subject reaches the door, entryway, or gate.

Turning now to the figures, example aspects are depicted with referenceto one or more components described herein, where components in dashedlines may be optional.

Referring to FIG. 1 , an access control system 100 is shown according toan aspect of the present disclosure. The access control system 100 caninclude a data communications device 105, a camera 110, an alarm device115 (e.g., notification device), and a processing circuit 120 that arecommunicatively coupled to each other via a communication bus 102.

In an aspect, the camera 110 can be an image capture device that uses animage sensor to capture images. Examples of the camera 110 may include astill camera or a video camera. The alarm device 115 can include anydevice that outputs an alert (e.g., visual output, audio output) basedon received information from the processing circuit 120.

The data communications device 105 may include wired or wirelessinterfaces (e.g., jacks, antennas, transmitters, receivers,transceivers, modems, wire terminals, etc.) for the access controlsystem 100 to conduct data communications with various systems, devices,or networks. For example, the communications interface may include anEthernet card and/or port for sending and receiving data via anEthernet-based communications network. The communications interface mayinclude a wireless transceiver (e.g., a WiFi transceiver, a Bluetoothtransceiver, a NFC transceiver, ZigBee, etc.) for communicating via awireless communications network. The communications interface may beconfigured to communicate via local area networks (e.g., a building LAN,etc.) and/or wide area networks (e.g., the Internet, a cellular network,a radio communication network, etc.) and may use a variety ofcommunications protocols (e.g., BACnet, TCP/IP, point-to-point, etc.).The data communications device 105 can be used to communicate with amobile device 170, which can execute an application for access control.

The processing circuit 120 can include a processor 125 and a memory 130.The processor 125 may be a general purpose or specific purposeprocessor, an application specific integrated circuit (ASIC), one ormore field programmable gate arrays (FPGAs), a group of processingcomponents, or other suitable processing components. The processor 125may be configured to execute computer code or instructions stored inmemory (e.g., fuzzy logic, etc.) or received from other computerreadable media (e.g., CDROM, network storage, a remote server, etc.) toperform one or more of the processes described herein. The memory 130may include one or more data storage devices (e.g., memory units, memorydevices, computer-readable storage media, etc.) configured to storedata, computer code, executable instructions, or other forms ofcomputer-readable information for performing operations corresponding toaccess control described herein. The memory 130 may include randomaccess memory (RAM), read-only memory (ROM), hard drive storage,temporary storage, non-volatile memory, flash memory, optical memory, orany other suitable memory for storing software objects and/or computerinstructions. The memory 130 may include database components, objectcode components, script components, or any other type of informationstructure for supporting the various activities and informationstructures described in the present disclosure. The memory 130 may becommunicably connected to the processor 125 via the processing circuit120 and may include computer code for executing (e.g., by processor) oneor more of the processes described herein.

The memory 130 can include various modules (e.g., circuits, engines) forcompleting processes described herein. For example, the memory 130 caninclude code or instructions for a facial recognizer 135, a database140, a face tracker 145, a duress detector 150, a skeletal tracker 155,a tailgate controller 160, and an encoder 165.

Referring to FIGS. 2-6 , methods of performing access control by theaccess control system 100 are described in conjunction with FIG. 1 .

Detecting Duress Using Facial Recognition and Skeletal Tracking

In some aspects, the access control system 100 can perform actions basedon receiving information from a subject (e.g., a person entering abuilding) indicating that the subject is accessing the building underduress. The access control system 100 can perform actions based onreceived information that does not provoke suspicion in anyonemonitoring the subject's movements.

In an aspect, the access control system 100 can use the camera 110 todetect (or capture) images regarding a subject (e.g., as the subjectapproaches a door or other access point), the processor 125 executes thefacial recognizer 135 that receives the images, the processor 125executes the skeletal tracker 155 that receives the images, the database140 maintaining facial information and skeletal information, and thealarm device 115.

The processor 125 executes the facial recognizer 135 to identify thesubject based on the received images. In an example, the facialrecognizer 135 can identify facial features based on the received images(e.g., expression, eye movements). The processor 125 executes theskeletal tracker 155 to identify movements of the subject based on theimages (e.g., movements of the head or limbs).

The database 140 can include facial information and movement informationregarding a plurality of subjects and registered gestures or facialexpressions. The database 140 can also include duress information whichincludes registered data representative of duress of each subject. Assuch, the facial recognizer 135 and skeletal tracker 155 can compare theidentified facial features and movements, respectively, compare theidentified features and movements to the duress information, and outputan indication of duress based on the comparisons.

The alarm device 115 can indicate that the subject is under duress basedon receiving the indication of duress from the facial recognizer 135and/or the skeletal tracker 155. In an example, the alert can includeone or more of a visual, text, or audio alert to alert others of theduress. In some examples, the alarm device 115 can present a name of thesubject under duress. In some examples, the alarm device can present alocation of the subject under duress based on a location of the camera110.

In some aspects, the facial recognizer 135 can detect a predeterminedemotion or other physiological response, such as fear, based on theimages. For example, the database 140 can include a fear threshold foreach subject. The facial recognizer 135 can compute a fear value basedon the received images, compare the fear value to the fear threshold forthe particular subject (e.g., the subject identified from the images),and responsive to the fear value being greater than the fear threshold,the access control system 100 can provide access while causing the alarmdevice 115 to indicate an alert indicating that the subject may beentering the building under duress.

In an aspect, the facial recognizer 135 can monitor an eye state of thesubject, such as eye movements, or eyes being open or closed. The facialrecognizer 135 can monitor a number of times the subject blinks theireyes as they approach the camera 110 based on the received images. Thedatabase 140 can include a predetermined number of blinks over a periodof time for each subject. In an example, one or more of the facialrecognizer 135 or the duress detector 150 can compare the monitorednumber of blinks to the predetermined number of blinks, and responsiveto the monitored number matching the predetermined number, the facialrecognizer 135 can determine the subject to be under duress and causethe alarm device 115 to indicate the alert accordingly.

In an aspect, the skeletal tracker 155 can monitor movement of thesubject based on the received images. The database 140 can includeskeletal information including one or more gestures associated with eachsubject that can be used to indicate duress. The gestures may be subtlemovements that the subject would not normally do when approaching thecamera 110, such as touching their face, touching their shoulder, ormoving their hand towards their pocket. In an example, one or more ofthe skeletal tracker 155 or the duress detector 150 can detect a gestureof the subject based on the received images, and based on the detectedgesture matching the gestures of the database, determine the subject tobe under duress and cause the alarm device 115 to indicate the alertaccordingly.

Referring to FIG. 2 , an example of a method 200 for detecting duressusing facial recognition and skeletal tracking is described herein.

At 202, the method 200 may include capturing an image regarding asubject. For example, the access control system 100 can use the camera110 to detect (or capture) images regarding a subject (e.g., as thesubject approaches a door or other access point). In an example, thesubject may include any worker, user, or person attempting to gainaccess to a facility, building, etc.

At 204, the method 200 may include identifying the subject based on thereceived images. For example, the access control system 100 may use theprocessor 125 to execute the facial recognizer 135 to receive the imagesand to identify the subject based on the received images. In an example,the facial recognizer 135 can identify facial features based on thereceived images (e.g., expression, eye movements). In an example, thefacial recognizer 135 may identify the subject by identifying facialfeatures in the images and comparing the facial features to facialinformation stored in the database 140. Facial information can includeinformation associated with facial features of one or more knownsubjects. Based on the comparison, the facial recognizer 135 maydetermine whether the facial features in the images match with any ofthe facial information of any of the known subjects. In an examples, thefacial recognizer 135 may determine a match based on a percentage offacial features of the images that have the most matches with facialinformation of a subject.

At 206, the method 200 may include identifying movements of the subjectbased on the images. For example, the access control system 100 may usethe processor 125 to execute the skeletal tracker 155 to identifymovements of the subject based on the images (e.g., movements of thehead or limbs). In an example, the skeletal tracker 155 may identifyskeletal features in the images and compare the skeletal features withskeletal information stored in database 140. Based on the comparison,the skeletal tracker 155 may determine whether the skeletal informationmatches (e.g., based on a percentage of matching) the skeletal featuresof the images.

At 208, the method 200 may include determining whether the subject isunder duress based on the comparisons. For example, the access controlsystem 100 can use the processor 125 to execute the duress detector 150to determine whether the subject is under duress based on thecomparisons by one or more of the facial recognizer 135 and the skeletaltracker 155. In an example, the duress detector 150 can determine duressbased one or more facial features and/or skeletal features/movementsindicating duress. For example, when performing the comparison, theduress detector 150 may compare the facial features and the skeletalfeatures (e.g., gestures—hand movements or signs, facial expressions) ofthe images to the facial information and skeletal information stored inthe database 140 which can include facial information regarding aplurality of subjects and registered gestures or facial expressions,including registered data representative of duress of each subjectand/or known general gestures or general facial features/expressionsindicative of duress. As such, the facial recognizer and skeletaltracker can compare the identified facial features and movements,respectively, compare the identified features and movements to theregistered data, and output an indication of duress based on thecomparisons.

At 210, the method 200 may include generating an alert based on thedetermination of duress. For example, the access control system 100 mayuse the alarm device 115 to indicate that the subject is under duressbased on receiving the indication of duress from the facial recognizer135 and/or the skeletal tracker 155. In an example, the alert caninclude one or more of a visual, text, or audio alert to alert others ofthe duress. In some examples, the alarm device 115 can present a name ofthe subject under duress. In some examples, the alarm device can presenta location of the subject under duress based on a location of the camera110.

Face Redaction Using Facial Recognition and Region of Interest Encoding

Facial recognition systems can be improved by redacting facialinformation from live video; however, this process can becomputationally intensive. The present solution addresses thesechallenges by using a CCTV video camera to redact faces in real timewith reduced computational requirements.

The access control system 100 can include the camera 110 that capturesimages, the facial recognizer 135 that can detect faces in an image, andthe encoder 165 that provides region of interest configuration. Regionof interest configuration for an encoder changes the configurationparameters for the encoder 165 so that an encoded frame has differentimage quality settings for user configurable regions within a frame. Ina typical system, this feature is used to enhance the image quality forfaces when face detection and recognition are running. However, in thepresent disclosure, this features is used to degrade image quality offaces.

The camera 110 can capture an image, which is provided to the facialrecognizer 135, and also to the encoder 165 to encode the image (e.g.,encode video of a stream of images). Facial detection can be executed onthe captured image to detect one or more areas (relating to regions ofinterest in the image which may include one or more detected faces)corresponding to detected faces. The facial recognizer 135 provides thedetected one or more areas to the encoder 165 to be configured as one ormore regions of interest. When the image is provided to the encoder 165for encoding, rather than enhancing the image quality in the regions ofinterest, the encoder 165 will degrade the encoding algorithm in theregions of interest (corresponding to the one or more areas receivedfrom the facial detection component) to redact the faces.

Referring to FIG. 3 , an example of a method 300 for face redactionusing facial recognition and region of interest encoding is describedherein.

At 302, the method 300 may include capturing an image regarding asubject. For example, the access control system 100 can include thecamera 110 to capture images regarding a subject.

At 304, the method 300 may include detecting a region of interest in theimage based on a detected face in the image. For example, the accesscontrol system 100 can include the processor 125 to execute the facialrecognizer 135 to detect faces in the image from the camera 110. In anexample, the facial recognizer 135 may identify the subject byidentifying facial features in the images and comparing the facialfeatures to facial information stored in the database 140. The facialrecognizer 135 may detect the region of interest based on the facialfeatures corresponding to facial information stored in the database 140.

At 306, the method can include encoding the image by degrading anencoding of the image in the region of interest. For example, the accesscontrol system 100 can include the processor 125 to execute the encoder165 to encode the image. In an example, the encoder 165 receives anindication of the region of interest from the facial recognizer 135. Theencoder 165 can change parameters for the region of interest so that anencoded frame has a lower image quality from other portions of theimage.

Intelligent Subject Detection

Facial recognition systems can be used to generate a real-time databaseof locations of subjects (e.g., people) within a building, such as toenable notification to a subject that they are required to be at aspecific location.

In an aspect, the access control system 100 can include a camera 110 andalarm device 115, including a plurality of audio speakers located ataccess points throughout a building; the facial recognizer 135 toidentify subjects as they pass access points throughout the building;the processor 125 (e.g., a server) that uses subject identificationinformation (e.g., locations of subjects based on facial recognition) togenerate the real-time database of locations; the data communicationsdevice 105 (e.g., a notification system) that can send notifications tothe mobile device 170; and a user interface application executing on themobile device 17) in communication with the access control system 100 tolocate subjects and provide alert messages to subjects.

In some aspects, the access control system 100 can be used when a firstsubject would like a second subject to be in a specific location, suchas to attend a meeting. A user interface application of the mobiledevice 170 (e.g., operated by the first subject) can receive a userinput indicating instructions to cause an alert message to be outputtedwhen the second subject is identified at an access point. The secondsubject moves through building as normal; when the access control system100 identifies the second subject at a particular access point, theaccess control system outputs the associated alert message (e.g., anaudio message, such as an audio message “Please attend the meeting inRoom 1” or “Please go to reception, there is a visitor waiting for you”)to one or more of the alarm device 115 or the mobile device 170.

In some aspects, the access control system 100 can be used to generatethe real-time database of locations of subjects (e.g., person mappingconfiguration). The access control system 100 can identify subjectsusing the facial recognizer 135 (e.g., when subjects are in the vicinityof cameras 110 of the access control system 100). In some aspects, suchas if the first subject would like to contact the second subject, theuser interface application of the mobile device 170 can receive a userinput indicating instructions to present the location of the secondsubject; the user interface application of the mobile device 170 mayalso receive instructions to cause the access control system 100 tooutput a notification and/or an alert message, such as but not limitedto an audio message, to the second subject to go to a particularlocation (e.g., the location of the first subject, which may correspondto the location of the mobile device 170 associated with the userinterface application which received the user input from the firstsubject).

Referring to FIG. 4 , an example of a method 400 for intelligent subjectdetection is described herein.

At 402, the method 400 can include detecting an image regarding asubject. For example, the access control system 100 can include aplurality of cameras 110 to detect an image regarding a subject.

At 404, the method 400 may include identifying a subject based on thedetected image. For example, the access control system 100 can includethe processor 125 to execute the facial recognizer 135 to detect facesin the image from the camera 110. In an example, the facial recognizer135 may identify the subject by identifying facial features in theimages and comparing the facial features to facial information stored inthe database 140.

At 406, the method 400 can include determining that a message stateassociated with the subject indicates a message of one or more messagesis assigned to the subject. For example, the processor 125 can determinethat the message state stored in the database 140 indicates a message ofone or more messages stored in the database 140 is assigned to thesubject.

At 408, the method 400 can include causing at least one of a mobiledevice associated with the subject or a notification device in avicinity of the subject to output the message. For example, theprocessor 125 may transmit (e.g., via communications device 105 or bus102) instructions to one or more of the alarm device 115 or the mobiledevice 170 to output the message.

In some examples, the method 400 may optionally include receiving afirst user input indicating instructions to present the correspondinglocation of the subject; and receiving a second user input, subsequentto the first user input, including the message, wherein the messageincludes information indicating that the subject is required to be at aspecific location different from the corresponding location of thesubject.

Intelligent Tailgating Detection

One of the issues encountered with typical access control systems is theproblem of tailgating. Tailgating occurs when a first person with validaccess to an area is closely followed by a second person who is notallowed access. When the first person is granted access, the secondperson also gains illegal access to the area by following the firstperson closely before a door or gate closes shut.

There are also cases where tailgating is allowed. A typical scenariowould be where an employee is chaperoning a number of visitors around abuilding. In this case, it may be valid that the visitors are allowedaccess to the building as long as they are accompanied by the employee.

At least one solution can address such concerns by using facialrecognition and other feature detection methods to determine whethertailgating that is occurring should be permitted. In some aspects, theaccess control system 100 includes the camera 110 to record video imagesof people approaching an access point; the alarm device including aspeaker system co-located with the camera 110 to allow audioannouncements and outputs an indication of a tailgating event havingoccurred; the facial recognizer 135 that uses video images from thecamera and stored images on the database 140 to grant access to an area;a face tracking system (e.g., facial recognizer 135 and/or face tracker145) that uses face detection and facial recognition to analyze imagesfrom the video camera and track the direction of travel of faces througha video image (e.g., to monitor for tailgating); the database 140maintaining face recognition information; and the tailgate controller160.

The database 140 can maintain, for each subject, a scheduler indicativeof whether to allow the subject to be granted or denied access to anarea at particular times during the day. Each scheduler entry caninclude a counter that identifies a number of additional subjects thatwill be allowed to tailgate the person when they enter the correspondingarea. This can be set to zero by default. The database 140 can maintainfacial features representative of each subject to facilitate facialrecognition. In some aspects, the database 140 maintain facial features(e.g., face entries) for persons of interest, that is, people who shouldbe denied access under particular circumstances or all circumstances.

The tailgate controller 160 can output announcements for when tailgatingis detected. The tailgate controller 160 can execute configurableactions for when tailgating is detected. For example, the tailgatecontroller 160 can cause the access control system 100 to deny access toeveryone; the tailgate controller 160 can cause the access controlsystem 100 to allow access and output an alarm via the alarm device 115.

Tailgating Detection

The access control system 100 can process images from the camera 110 togenerate a data structure maintaining location and direction of movementof all faces (e.g., all subjects associated with respective faces) inthe received images. The data structure may be maintained in thedatabase 140. The facial recognizer 135 and/or the face tracker 145 canexecute facial recognition to identify each subject, and determinewhether to allow access to the identified subject based on the facialrecognition. In some aspects, tailgating detection includes determiningone or more of a location, a direction, or a speed of travel for eachface in a field of view (e.g., each face associated with subjectsadditional to a first identified subject). The facial recognizer 135and/or the face tracker 145 can determine which subjects(s)corresponding to the identified faces are allowed access through thecorresponding access point. If the facial recognizer 135 and/or the facetracker 145 determines that each subject is allowed access, the facialrecognizer 135 and/or the face tracker 145 grants access as there is norisk of tailgating. If the facial recognizer 135 and/or the face tracker145 determines that at least one tracked face is not identified tocorrespond to a known subject (e.g. maintained in the database 140) or asubject that is not allowed access, the face tracker 145 can use thelocation, direction, and/or speed information to determine whether suchsubjects will be able to enter the access point at the same time as thesubject that is allowed access. As such, the face tracker 145 candetermine whether there is a risk of tailgating.

Responsive to determining that there is no risk or likelihood oftailgating, the tailgate controller 160 can allow access to the firstsubject. Responsive to determining that there is a risk or likelihood oftailgating, the tailgate controller 160 can retrieve information fromthe database 140 regarding whether the first subject is expected to bewith other subjects (e.g., in a schedule associated with the subject).For example, the database 140 can maintain information regarding whetherthe first subject is expected to be accessing the particular accesspoint at the particular time period, and whether the first subject willbe accompanied by one or more unknown subjects; if the schedule doesindicate that these conditions are met, then the system can allow accessto the first subject. Similarly, responsive to determining that theseconditions are not met (e.g., the first subject is not expected to beaccessing the particular access point at the particular time period oris not expected to be accompanied) the tailgate controller 160 candetermine tailgating to have occurred.

Tailgating Notification

Responsive to determining tailgating to have occurred, the tailgatecontroller 160 can execute various actions. The tailgate controller 160can deny access to the first subject. The tailgate controller 160 cancause the alarm device 115 to output an indication of the tailgating,such as an audio announcement or other alarm. The audio announcement mayindicate that subjects not allowed to access should move further fromthe access point; it will be appreciated that the facial recognizer 135,the face tracker 145, and/or the tailgate controller 160 canperiodically process the received images to determine whether tailgatingis occurring, such that if subjects determined to be tailgating in afirst image have moved further from the access point by the time asecond, subsequent image is analyzed, the tailgate controller 160 candetermine based on the second image that tailgating is not occurring anddeactivate the alarm or otherwise discontinue actions that would takeplace based on tailgating. In some aspects, the alarm device 115 canoutput an alert signal indicating there is a risk of tailgating. In someaspects, the tailgate controller 160 can allow access to the firstsubject and each of the addition subjects, and the alarm device 115 canoutput an alarm. Outputting an alarm can include outputting an audioalert to indicate that tailgating has been detected.

Subjects of Interest

The access control system 100 can maintain subject of interestinformation. Responsive to detecting a subject of interest in thereceived images, the access control system 100 can output an alarmindicating the name and/or location of the subject of interest; outputan audible alarm on the speaker adjacent to the camera 110; and/or denyaccess to each subject until the person of interest leaves the scene(e.g., is no longer detected in the received images).

Referring to FIG. 5 , an example of a method 500 for managingintelligent tailgating is described herein.

At 502, the method 500 can include detecting an image regarding asubject. For example, the access control system 100 can include thecamera 110 to capture images regarding a subject.

At 504, the method 500 may include identify the subject based on theimage and identifying at least one additional subject based on theimage. For example, the access control system 100 can include theprocessor 125 to execute the facial recognizer 135 to detect faces inthe image from the camera 110. In an example, the facial recognizer 135may identify the subject by identifying facial features in the imagesand comparing the facial features to facial information stored in thedatabase 140.

At 506, the method 500 can include determining movement informationregarding the subject and the at least one additional subject. Forexample, the processor 125 may execute the face tracker 145 to determineone or more of a location, a direction, or a speed of travel for eachface corresponding to the plurality of subjects identified by the facialrecognizer 135. In an example, the face tracker 145 may monitormovements of each of the faces based on the images and calculatemovements based on changes to locations of the face between images.

At 508, the method 500 can include determining whether the at least oneadditional subject has permission to move through an access point basedon the identified subject, the identified at least one additionalsubject, and the movement information. For example, the processor 125may execute the face tracker 145 to determine that at least one trackedface is not identified to correspond to a known subject (e.g. maintainedin the database 140) or a subject that is not allowed access, the facetracker 145 can determine and use the location, direction, and/or speedinformation to determine whether such subjects will be able to enter theaccess point at the same time as the subject that is allowed access. Assuch, the face tracker 145 can determine whether there is a risk oftailgating.

In some aspects, the method 500 can optionally include providing accessto the one or more subjects based on the determination. For example, theprocessor 125 may execute the tailgate controller 160 to provide accessto one or more of the subjects. In an example, responsive to determiningthat there is no risk or likelihood of tailgating, the tailgatecontroller 160 can allow access to the first subject. Responsive todetermining that there is a risk or likelihood of tailgating, thetailgate controller 160 can retrieve information from the database 140regarding whether the first subject is expected to be with othersubjects (e.g., in a schedule associated with the subject).

The method 500 may also optionally include alerting others about thetailgating. For example, the alarm device 115 may alert others of thetailgating based on a signal from one or more of the facial recognizer135 or the face tracker 145.

Second Factor Authentication Using Mobile Devices

Access control systems may not always identify a subject, even iffeatures regarding the subject are maintained in a database. Althoughfacial recognition systems can be highly accurate, there can be a chance(however small) that the system may fail to identify a known subject.This may be due to an unclear image captured by the face recognitionsystem or some subtle changes to the subject's appearance (e.g.hairstyle) or due to the subject wearing a covering over their face(face obscuration). The present solution can use a mobile device toprovide second (e.g., backup) information to identify and/orauthenticate the subject.

The access control system 100 can include the camera 110 that capturesimages for the purposes of person identification; the datacommunications device 105 (e.g., a WiFi and/or Bluetooth transceiver);the facial recognizer 135 that identifies subjects using the capturedimages; and the database 140 including features regarding subjects suchas images of each subject's face, as well as information regarding themobile device 170 corresponding to each subject, such as an identifierof the mobile device 170. In some aspects, the mobile device 170executes an application that can communicate with the access controlsystem 100.

Responsive to not identifying a subject in the received images, theaccess control system 100 can execute various actions as backup facerecognition, including using codes, mobile devicebased facialrecognition, and/or mobile device information.

Codes

The access control system 100 can generate a code, such as animage-based code (e.g., a QR code). The code may be a single use code(e.g., the code is newly generated for each authentication). The accesscontrol system 100 can transmit the code to the application executed bythe mobile device 170, such that the application presents the code fordisplay on the mobile device 170. The access control system 100 canreceive an image from the camera 110 that includes the code (e.g., ifthe subject positions the code in the field of view of the camera).Responsive to the code identified in the image matching the transmittedcode, the access control system 100 can allow access.

The access control system 100 can transmit instructions to theapplication on the mobile device 170 to cause the application to executea facial recognition process. The application can receive results of thefacial recognition process, and transmit the results to the accesscontrol system 100. The access control system 100 can also receive alocation where the facial recognition was performed (e.g., GPS data)and/or the image(s) captured by the mobile device 170 based on which themobile device 170 executed facial recognition. The access control system100 can compare the location to a known location of the access point toverify that the locations match, and can similarly compare details in abackground of the image(s) captured by the mobile device 170 to knownfeatures of the environment around the access point to determine whetherthe locations match. Responsive to one or both matches being detected,the access control system 100 can grant access.

Mobile Device Information

Responsive to not detecting the person using facial recognition, theaccess control system 100 can request, from the mobile device 170,mobile device identification information, such as a media access control(MAC) address or wireless identifier (e.g., Bluetooth identifier). Theaccess control system 100 can receive the mobile device identificationinformation and compare this information to information maintained inthe database 140, and responsive to determining a match, the accesscontrol system 100 can allow access.

Referring to FIG. 6 , an example of a method 600 for second factorauthentication in an access control system is described herein.

At 602, the method 600 can include detecting an image regarding asubject. For example, the access control system 100 can include thecamera 110 to detect an image regarding a subject.

At 604, the method 600 may include executing an identification algorithmto identify the subject based on the detected image and the facialinformation maintained by a database. For example, the access controlsystem 100 can include the processor 125 to execute the facialrecognizer 135 to detect a face in the image from the camera 110. In anexample, the facial recognizer 135 may identify the subject byidentifying facial features in the image and comparing the facialfeatures to facial information stored in the database 140.

At 606, the method 600 may include determining that the identificationalgorithm was not successful in identifying the subject. For example,the processor 125 may execute the facial recognizer 135 to determinethat the identification algorithm was not successful in identifying thesubject based on facial features of the image not matching facialinformation stored by the database 140.

At 608, the method 600 may include performing one or more second factorauthentication processes based on the identification algorithm beingunsuccessful. In an example, the processor 125 and/or the datacommunications device 105 can perform the second factor authenticationprocess. In some examples, the second factor authentication process caninclude outputting a code to the mobile device 170 such that the codecan be presented to the camera 110 for matching to the outputted codeand granting access via an access point. In some examples, the secondfactor authentication process can include comparing a location of themobile device 170 to a location of the access point to grant access viathe access point. In some examples, the second factor authenticationprocess can include receiving an identifier from the mobile device 170and determining whether to grant access via the access point based onthe identifier.

The construction and arrangement of the systems and methods as shown inthe various exemplary aspects are illustrative only. Although only a fewaspects have been described in detail in this disclosure, manymodifications are possible (e.g., variations in sizes, dimensions,structures, shapes and proportions of the various elements, values ofparameters, mounting arrangements, use of materials, colors,orientations, etc.). For example, the position of elements can bereversed or otherwise varied and the nature or number of discreteelements or positions can be altered or varied. Accordingly, all suchmodifications are intended to be included within the scope of thepresent disclosure. The order or sequence of any process or method stepscan be varied or re-sequenced according to alternative aspects. Othersubstitutions, modifications, changes, and omissions can be made in thedesign, operating conditions and arrangement of the exemplary aspectswithout departing from the scope of the present disclosure.

The present disclosure contemplates methods, systems and programproducts on any machine-readable media for accomplishing variousoperations. The aspects of the present disclosure can be implementedusing existing computer processors, or by a special purpose computerprocessor for an appropriate system, incorporated for this or anotherpurpose, or by a hardwired system. Aspects within the scope of thepresent disclosure include program products comprising machine-readablemedia for carrying or having machine-executable instructions or datastructures stored thereon. Such machine-readable media can be anyavailable media that can be accessed by a general purpose or specialpurpose computer or other machine with a processor. By way of example,such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROMor other optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to carry or storedesired program code in the form of machine-executable instructions ordata structures and which can be accessed by a general purpose orspecial purpose computer or other machine with a processor. Combinationsof the above are also included within the scope of machine-readablemedia. Machine-executable instructions include, for example,instructions and data which cause a general purpose computer, specialpurpose computer, or special purpose processing machines to perform acertain function or group of functions.

References to “or” may be construed as inclusive so that any termsdescribed using “or” may indicate any of a single, more than one, andall of the described terms. References to at least one of a conjunctivelist of terms may be construed as an inclusive OR to indicate any of asingle, more than one, and all of the described terms. For example, areference to “at least one of ‘A’ and ‘B’ can include only ‘A’, only‘B’, as well as both ‘A’ and ‘B’. Such references used in conjunctionwith “comprising” or other open terminology can include additionalitems.

What is claimed is:
 1. An access control system, comprising: a camera configured to record an image of two or more subjects approaching an access point; and a processor in communication with the camera and configured to: identify a subject based on the image; identify at least one additional subject based on the image; determine movement information regarding the subject and the at least one additional subject; determine whether the at least one additional subject has permission to move through an access point based on the subject, the at least one additional subject, and the movement information; and permit or deny access movement of the subject or the at least one additional subject through the access point based on whether the at least one additional subject has permission.
 2. The access control system of claim 1, further comprising: a database configured to store a scheduler corresponding to the subject, wherein the scheduler includes scheduling information indicating a time the subject has permission to move through the access point, wherein the processor is further configured to determine whether the at least one additional subject has permission to move through the access point further based on the scheduler.
 3. The access control system of claim 2, wherein the scheduler further includes a counter identifying a number of additional subjects having permission to move through the access point with the subject, wherein the processor is further configured to determine whether the at least one additional subject has permission to move through the access point further based on the counter.
 4. The access control system of claim 1, wherein the processor is further configured to: determine a likelihood of risk of tailgating by the at least one additional subject; and determine whether the at least one additional subject has permission to move through the access point further based on the likelihood of risk of tailgating.
 5. The access control system of claim 1, wherein to determine the movement information, the processor is further configured to determine the movement information regarding the subject and the at least one additional subject based on one or more of a location, a direction, or speed information of at least one of the subject or the at least one additional subject.
 6. The access control system of claim 1, further comprising: an alarm device configured to output an indication of a tailgating event based on the at least one additional subject not having permission.
 7. The access control system of claim 1, wherein the processor is further configured to: determine the subject has permission to move through the access point; and deny movement by the subject through the access point based on whether the at least one additional subject has permission.
 8. A method comprising: receiving, from a camera, an image of two or more subjects approaching an access point; identifying a subject based on the image; identifying at least one additional subject based on the image; determining movement information regarding the subject and the at least one additional subject; determining whether the at least one additional subject has permission to move through an access point based on the subject, the at least one additional subject, and the movement information; and permitting or denying access movement of the subject or the at least one additional subject through the access point based on whether the at least one additional subject has permission.
 9. The method of claim 8, further comprising: storing scheduling information indicating a time the subject has permission to move through the access point; and determining whether the at least one additional subject has permission to move through the access point further based on the scheduling information.
 10. The method of claim 9, wherein the scheduling information includes a counter identifying a number of additional subjects having permission to move through the access point with the subject, and further comprising: determining whether the at least one additional subject has permission to move through the access point further based on the counter.
 11. The method of claim 8, further comprising: determining a likelihood of risk of tailgating by the at least one additional subject; and determining whether the at least one additional subject has permission to move through the access point further based on the likelihood of risk of tailgating.
 12. The method of claim 8, further comprising: determining movement information regarding the subject and the at least one additional subject based on one or more of a location, a direction, or speed information of at least one of the subject or the at least one additional subject.
 13. The method of claim 8, further comprising: outputting, via an alarm device, an indication of a tailgating event based on the at least one additional subject not having permission.
 14. A non-transitory computer-readable device having instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising: receiving, from a camera, an image of two or more subjects approaching an access point; identifying a subject based on the image; identifying at least one additional subject based on the image; determining movement information regarding the subject and the at least one additional subject; determining whether the at least one additional subject has permission to move through an access point based on the subject, the at least one additional subject, and the movement information; and outputting, via an alarm device, an indication of a tailgating event based on the at least one additional subject not having permission.
 15. The non-transitory computer-readable device of claim 14, wherein the operations further comprise: storing scheduling information indicating a time the subject has permission to move through the access point; and determining whether the at least one additional subject has permission to move through the access point further based on the scheduling information.
 16. The non-transitory computer-readable device of claim 15, wherein the scheduling information includes a counter identifying a number of additional subjects having permission to move through the access point with the subject, and the operations further comprising: determining whether the at least one additional subject has permission to move through the access point further based on the counter.
 17. The non-transitory computer-readable device of claim 14, wherein the operations further comprise: determining a likelihood of risk of tailgating by the at least one additional subject; and determining whether the at least one additional subject has permission to move through the access point further based on the likelihood of risk of tailgating.
 18. The non-transitory computer-readable device of claim 14, wherein the operations further comprise: determining movement information regarding the subject and the at least one additional subject based on one or more of a location, a direction, or speed information of at least one of the subject or the at least one additional subject.
 19. The non-transitory computer-readable device of claim 14, wherein the operations further comprise: permitting or denying access movement of the subject or the at least one additional subject through the access point based on whether the at least one additional subject has permission.
 20. The non-transitory computer-readable device of claim 14, wherein the operations further comprise: determining the subject has permission to move through the access point; and denying movement by the subject through the access point based on whether the at least one additional subject has permission. 